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ABSTRACT 



Information from tests, when considered in the 
context of decision theory, can play an important role in vocational 
development. This role is primarily one of stimulating and 
facilitating exploratory behavior. Objective procedures for 
converting test data into counseling information are discussed and 
illustrated with special attention being given to similarity scores 
and similarity score profiles. The similarity score profiles, when 
used in conjunction with success estimates, avoid many of the 
pitfalls inherent in Parsonian approaches to test interpretation. A 
computer-based system incorporating the above data- inf ormation 
conversion procedures was developed and field tested with potential 
vocational school students. Longitudinal validation analyses 
involving more than 1,500 students and 20 interest and aptitude 
measures provided the empirical base for the data-inf or ma tion 
conversion procedures. Similarity scores and success estimates were 
developed for each of the 12 vocational program areas in which the 
students enrolled. These reporting procedures were then field * ^ted 
with an additional group of 900 students enrolled in 12 " ols. 

Illustrations of the similarity score reporting procedui. mu 
student reactions to them are provided. (Author/CK) 
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ABSTRACT 



Part 1 : Rationale 

Information from tests, when considered in the context of decision theory, can play an 
important role in vocational development. This role is primarily one of stimulating and 
facilitating exploratory behavior. Objective procedures for converting test data into counseling 
information are discussed and illustrated with special attention being given J o similarity 
(centour) scores and similarity score profiles. Alternative names emphasizing the guidance use 
of these procedures are suggested, i.e., exploration indices and exploration maps. The 
similarity score profiles, when used in conjunction with success estimates, avoid many of the 
pitfalls inherent in Parsonian approaches to test interpretation. 



Part 2: Field Trial with Feedback 



A computer-based system incorporating the above data-information conversion procedures 
was developed arid field tested with potential vocational school students. Longitudinal 
validation analyses involving more than 1,500 students and 20 interest and aptitude measures 
provided the empirical base for the data-information conversion procedures. Similarity scores 
and success estimates were developed for each of the 12 vocational program areas in which the 
students enrolled. These reporting procedures were then field tested with an additional group 
of 900 students enrolled in 12 high schools. Illustrations of the similarity score reporting 
procedures and student reactions to them are provided. 
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CONVERTING TEST DATA TO COUNSELING INFORMATION 1 



Dale J. Prediger 



PART 1: RATIONALE 



This report is concerned with objective pro- 
cedures for converting test scores and other data 
into information which is relevant to a coimselee's 
educational and vocational plans, decisions, or 
problems. Validity studies are crucial to data- 
information conversion procedures. However, these 
studies as usually reported, are of little direct help 
to the counselor who has a student and a s -t of test 
scores before him. While it is true that an accumu- 
lation of validity studies performed within a 
theoretical framework may support various uses of 
a particular test, the task of converting a 



counselee's test scores into usable information is 
left undone. Typically, the counselor can find the 
standing of a counselee in some norm group; after 
that, he is on his own. Professional knowledge, 
clinical judgment, end personal sensitivity always 
will play crucial roles in test interpretation. 
However, objective datahnformation conversion 
procedures can make the counselor's job much 
easier. Just what does a percentile rank of 63 on 
the XYZ Mechanical Aptitude Test say to the 
student and his counselor? 



Local Validity Data and Decision-Making 



Almost 15 years have passed since Dyer (1357) 
made a convincing case for local studies of test 
validity. Dyer was only one of a host of measure- 
ment specialists who cautioned test users about 
accepting tests on the basis of face validity or 
assuming that one o- two validity studies con- 
ducted in some other setting provide sufficient 
evidence that a test wculd be useful in their setting 
and for their purposes. Recent reviews of 
validation research (Gh ^elli, 1966; Prediger, Waple, 
& Nusbaum, 1968) have reinforced this caution. 

An excellent discussion of the role of local 
validity data in vocational guidance and decision- 
making has been provided by Clarke, Gelatt, and 



Levine (1965). In their discussion, attention was 
focused on the process of decision-making, with 



1 With minor modifications. Part 1 of this report will appear as an 
article in Volume 1 8 of the Journal of Counseling Psychology under 
the title "Data-lnformation Conversion Procedures in Test Inter- 
pretation. " Part 2, also with minor modifications, will appear as an 
article in the Journal of Educational Measurement under the title 
"Converting Test Oata to Counseling Information: System Trial— 
with Feedback," 

The study reported in Part 2 of this reoort was partially supported 
by a contract with the Office of E’dncation, U.S. Department of 
Health, Education, and Welfare. Support was also provided by the 
Penta-County Vocational High School (Perrysburg, Ohio) and the 
University of Toledo, 



>rmation on the possible outcomes of various 
rses of action being seen as a necessary if not 
icient condition for wise decisions. Examples 
ocal validity studies were given to illustrate the 
elopment of objective probabilities useful in 
cational planning. As with Dyer, use of 
erience (expectancy) tables was emphasized, 
sequent articles (Gelatt & Clarke, 1967; Katz, 
>6, 1969; Thoresen & Mehrens, 1967) elabo- 
id on the role of objective probabilities in 
; i sion-making, the influence of objective 
babilities on subjective probabilities, and the 
factions among subjective probabilities, choice 
ion utilities, and personal values. 



Bridges between 



Goldman (1961) described three objective 
dgjes between test scores and their meaning for 
i counselee: the norm bridge, the regression 
dge, and the discriminant bridge. Most of our 
rent data-information conversion procedures 
isist of some form of the norm bridge. As 
Idman noted, the norm bridge is an incomplete 
dge since test norms simply permit one to 
imate standing in some group and do not, per se, 
licate the implications of this st; . '’hr 
jression bridge, however, is a complete bridge 
>m test scores to their implications and, as such, 
idily lends itself to data-information conversion, 
ually,, the implications are in the form of success 
adictiions obtained v a experience tables or 
jressuon equations, 

Trie third bridge nested b\ Goldman, the dis- 



Katz (1963, 1966), in particular, was careful to 
show how the decision-making process is related to 
the broader process of vocational development. 
Results from the massive Project TALENT valida- 
tion studies also have been placed firmly within the 
context of vocational development theory and 
decision theory (Cooley & Lohnes, 1968). We have 
passed the era in which the Parsonian concept of 
test interpretation could be viewed as the epitome 
of educational and vocational guidance. However, 
the above studies and formulations leave little 
doubt about the continued importance of test 
information, when properly validated, in the 
vocational-development process. 



and Information 



criminant bridge, provides an objective measure of 
a counselee's similarity to various criterion groups. 
Discriminant analysis techniques, when combined 
with the similarity score procedures developed by 
Tiedeman, Bryan, and Rulon (1951), permit the 
comparison of a counselee's test results with those 
of various criterion groups along the major dimen- 
sions of test data that di+ferentir *e the groups. The 
complen. nt~; ( natur. t iarity rnd sUwCess 

estimates was first discussed 20 years ago (Rulon, 
1951; Tiedeman et al., 1951). Few counselors, 
however, are familiar with the characteristics of 
similarity (centour) scores or their pcrc-ntial role in 
test interpretation. These topics have T&c^ ? ved little 
attention in testing texts or test Interpretation 
manuals. For this reason, an illustration is provided 
in the following section. 



Data-information Conversion via Similarity Scores 



Consider the information n^eds of Fred, n high 
hooi student thinking about enrobing in a 
jcational-techmcal education program. Sirrcherity 
ore procedures appliec to F red's high school 
ade record and test scores could result in a report 



indicating Fred's similarity to succes^ui and satis- 
fied students in various programs. Ir the example 
that follows, the similarity scores shrown in paren- 
theses after each of the vocational program^ are on 
a scale ranging from 0 to 100 with 00 represen- 
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:he highest degree of similarity. The closer 
5 scores on the relevant tests are to the test 
> of the typical successful and satisfied stu- 
in a vocational program, the higher his 
rity score will be for that program. Fred's 
irity score report might look like this: voca- 
I horticulture (87), carpentry (41), com- 
a! art (28), auto body (26), distributive 
ition (25), auto mechanics (14), radio-TV 
' (3), and data processing (1). 
us, on the basis of the measures used, Fred's 
jdes and interests are most similar to students 
cational horticulture. Carpentry ranks second, 
hree other programs are in an approximate tie 
nird. Fred is least similar to students in data 
issing and radio-TV repair, 
this example, test data have been transformed 
information that is directly relevant to one of 



the major functions of tests in education^ and 
vocational gu idance— facilitating exploratory 

behavior. Fred's counselor might use the similarity 
scores quite advantageously in stimulating Fred to 
explore the program options available to him. For 
this reason, similarity scores might be more 
appropriately called exploration indices; that is, 
indices which suggest choice options a counselee 
may want to explore. 

Of course, the similarity scores should not be 
used alone in facilitating exploratory behavior. 
Their potential value in this example lies in 
suggesting vocational possibilities that might not 
have been recognized otherwise. The degree to 
which Fred explores these possibilities will be a 
function of his value system and the opportunities 
provided him. 



Complementary Role of Success Estimates 



ccess estimates obtained from regression 
fses or expectancy tables also can be used to 
tate e-^pL "rtory behavior. For example, Fred 
t be encouraged to explore the vocational 
rams for which he is predicted to receive the 
jst grades. However, estimates of success might 
lore appropriately incorporated into the actual 
oration process where they can be considered 
onjunction with a host of other relevant 
>rs. After all, Fred may not place much value 
naking high grades. His similarity scores could 
tify vocational programs in which he is similar 
"successful and satisfied" enrollees. His 
able level of success could then be determined 
i further exploration. Thus, a two-stage 
egy is suggested with similarity scores being 
to stimulate and facilitate exploration and 
?ss estimates being an important consideration 
ig the process of exploration. 
ther considerations also indicate the need for 
ion in the use of success estimates as the 
ary basis for facilitating exploration. Consider, 
example, an experience table showing the 
ionship between the scores on some test and 
es in carpentry. Could this table be used 



appropriately with Sally, Fred's sister? Can Sally 
be considered similar to the group from which the 
experience table data were obtained? To what 
degree would the trends shown in the results apply 
to her? Similarly, to what degree would success 
predictions in radio-TV repair apply to Fred 
(similarity score = 3)? Can we legitimately use 
Fred's test scores to predict lab grades in cos- 
metology? Or, in another context, are we justified 
in comparing a high school senior's college fresh- 
man GPA predictions in engineering, humanities, 
education, physical science, or business? These 
questions, recently discussed by Rulon, Tiedeman, 
Tatsuoka, and Langmuir (1967), need further 
investigation. A "reasonable" degree of similarity 
between counsolees and the validation sample 
might well be an appropriate prerequisite for the 
use of success estimates in counseling. 

A second difficulty with success predictions 
results from the well-known "criterion problem." 
Obtaining a suitable criterion of success in educa- 
tion and training is difficult enough, but when one 
moves into the world of work, the definition and 
measurement of success become infinitely more 
complex (Thorndike, 1963; Thorndike & Hagen, 




applied.) This is all that is needed to permit use of 
similarity score procedures. Thus, data-information 
conversion would be possible. 



969). Members of various occupations or occupa- 
onal clusters can be readily identified, however. 
3ross criteria for identifying "successful and 
atisfied" occupational group members can also be 



Overcoming the Profile Problem 



Judgment of a counselee's similarity to various 
ritericn groups is not new in counseling. Certain 
ommercially available inventories, e.g., the Strong 
Vocational Interest Blank and the Kuder Occupa- 
ional Interest Survey, provide scores giving direct 
id in the process. However, special test construe- 
ion and criterion group requirements present 
ormidable barriers to the development of such 
inventories, In addition, existing inventories cover 
>nly one area of measurement, usually interests, 
iimultaneous coverage of other areas (e.g., 
ptitudes, aspirations, biographical data) is not 

ctica! through use of these special test construe- 
jh techniques. 

Profiles showing the typical performance of 
various criterion groups often have been provided 
or other tests and inventories. Counselors are 
jxpected to use these profiles by noting the 
iimilarity of a eounseiee's profile to the profiles of 
:he criterion groups. Anyone who has engaged in 
;his process does not need a description of what 
las long been known as the profile problem 
Tiedeman, 1 954). 

Similarity scores take the guesswork out of 
Drofile comparison; but unless they are used in 
conjunction with discriminant analysis procedures, 
they fail to deal directly with important aspects of 
the profile problem, e.g., do the criterion groups 
actually score differently on the measures 
involved? If so, what are the important measures 
and how should they be weighted? Unfortunately, 
a set of similarity scores can be obtained from 
purely irrelevant variables. 

For the above reasons, discriminant analysis 
procedures are often used in conjunction with 
similarity scores. These procedures enable one to 
determine whether the criterion groups are, in fact, 
differentiated by the measures. If so, the measures 
can be weighted and combined into independent 



factors (discriminant functions) that maximize 
criterion group differentiation. Furthermore, the 
number of factors needed for criterion group 
differentiation can be determined. Thus, one may, 
and usually will, find that two factors account for 
most of the discriminatory power of a set of 
measures when applied to criterion groups relevant 
to educational and vocational counseling. Finally, 
the nature of the factors can be identified by 
noting the measures that correlate highly with 
them. 

Equations resulting from discriminant analysis 
can be used to calculate criterion group positions 
on the discriminating factors. These positions, in 
turn, nan be plotted on a coordinate grid with the 
vertical and horizontal axes representing the two 
major factors. In the same manner, the factor 
scores of a given student may be calculated and 
plotted. The student's position on the coordinate 
grid then may be compared visually with the 
criterion group positions. 

This technique for graphically depicting a stu- 
dent's similarities and dissimilarities was implicit in 
early work on the profile problem (Tiedeman, 
1954) and was specifically suggested more than 10 
years ago (Whitla, 1957; Dunn, 1959). However, it 
has been mentioned only occasionally in the 
professional literature since then (e.g., Baggaley & 
Campbell, 1967; Cooley & Lohnes, 1968) and has 
received little attention in testing texts, Rulon and 
his colleagues (1967) have provided a detailed 
discussion of the rationale underlying discriminant 
analysis and similarity score procedures along with 
ample illustrations of the resulting graphical 
solution to the profile problem. However, in 
presenting the illustrations, emphasis was placed on 
representing the geometry of the statistical 
procedures. Little attention was given to test 
interpretation applications of the illustrations. 
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Data-lriformation Conversion via "Similarity Score Profiles" 



ie graphical procedures described above can 
? several important functions in data- 
mation conversion. Application of these 
2 dures to the problem faced by Fred and his 
selor— consideration of vocational program 
>ns— is illustrated by the similarity score 
le presented in Figure 1. Note that the two 
nsions or factors that best differentiate the 
tiona! programs are represented by the axes of 
coordinate grid. Each factor is scaled with a 
1 of 50 and a standard deviation of 10 for 
;nts in all vocational programs combined. Of 
10 aptitude and interest measures used in the 
'sis on which the profile is based, those having 
ir correlations with an absolute value of .40 or 
f are listed as factor anchors. (The following 
)ols are used to indicate level of correlation: 
M > .59; * =_r of .50 * .59; no * =x of .40 - 

ed's factor scores of 56 and 36 for Factors 1 
2, respectively, have been plotted on the 
dinats grid along with the mean scores of the 
: vocational programs. (In several instances, 
auto body and welding, related vocational 
rams have been combined into one group.) 
ellipse surrounding the vocational horticulture 
ip encloses the factor scores of approximately 
of the students in the group and is analogous 
ne scatter diagrams often seen in discussions of 
elation. Similar, but not identical, ellipses 
d be used to represent the scatter of factor 
es for the other vocational programs. One 
se should be sufficient, however, to obtain a 
i estimate of program overlap, 
he size of Fred's similarity scores is reflected in 
distances between Fred and the various voca- 
al programs on the profile. Although the 
tionship of similarity score size to distance is 
exact, it is sufficiently high to give a useful 
al picture of degree of similarity. For example, 
j's position is quite close to that of the 
ational horticulture group, the group for which 
eceived his highest similarity score, i.e., 87. On 
other hand, programs for which he has low 
larity scores are much farther away, 
y comparing Fred's position on the profile 



with the positions of the various vocational pro- 
grams, one can obtain valuable insights into the 
reasons underlying Fred's similarity scores. Field 
trials with potential vocational school students 
(Prediger, 1970) have shown this is important, 
since use of similarity scores by themselves often 
leaves the counselee with the question, "But why 
did the scores come out like that?" In Fred's case, 
we see that both he and the typical horticulture 
student score only slightly above average on Factor 
1, while Factor 2 scores suggest relatively strong 
artistic interests and relatively low mechanical 
reasoning aptitude and scientific interests. 

Fred may wonder why his similarity score for 
area H was so low. From the profile, we see that 
the Factor 2 scores for Fred and area H students 
are relatively far apart. In addition, the position of 
area H students on Factor 1 suggests that they have 
stronger clerical skills and interests than Fred. 
Similar reasoning can be followed in comparisons 
with the other groups represented on the profile. 

In a sense, similarity score profiles provide the 
student with a map of an important domain of 
choice options. In helping Fred understand the 
possible reasons underlying his similarities and 
dissimilarities, one is helping him to project certain 
aspects of his self into the choice domain. His 
position in that domain is shown, and he is 
encouraged to relate his position and character- 
istics to those of students who have elected various 
choice options. This might be viewed as a type of 
vicarious exploration within the dimensions 
mapped by the similarity score profiles. Perhaps a 
more appropriate, guidance-oriented term for these 
profiles would be "exploration maps. " 

Variation in the characteristics of students who 
have made a specific choice will be clearly evident 
from the ellipse shown on the profile (or explora- 
tion map). If ellipses are shown for all of the 
vocational programs, overlap among the various 
groups also will be evident. Criticisms of counseling 
applications of trait and factor research sometimes 
are based on the fact that divergencies within 
criterion groups and similarities among criterion 
groups are concealed. Similarity score profiles 
reveal these facts of life. At the same time. 
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Fig. 1. Similarity Score Profile for Fred Cartesian. 
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mces among criterion groups, if present in 
ita, are presented in a manner that permits 
Dunselee to "try the various groups on for 

;hould be noted that an extreme score on 
ne of the variables shown as factor anchors 
by i + self, have a major influence on Fred's 
scores. Hence, Fred's test score reports 
i be consulted while discussing his similarity 
profile. (Fred's counselor would probably 



want to do this anyway in order to facilitate Fred's 
participation in the test interpretation process.) 
The factor positions of the vocational programs, on 
the other hand, are unlikely to be subject to the 
influence of extreme scores on one or two vari- 
ables. These positions are, after all, based r,i group 
means rather than the scores of a singie individual, 
it is well-known that the spread of group means on 
a variable will be much less than the spread of 
individual scores. 



Similarity and the Status Quo 



ce the reasons underlying Fred's similarity 
; have been determined, Fred and his coun- 
may be able to develop a program of activities 
tudy that will increase his similarity score for 
n criterion group. The feasibility of doing this 
j, of course, depend on the variables involved, 
fver, the suggested strategy represents an 
pie of a counseling application of test data 
facilitates change in the status quo rather than 
y a representation of it. 

lother strategy can be used in combating the 
• quo, in this case, a strategy that reflects 
utional goals. Suppose a college is unhappy 
the characteristics of members of certain 
;u!ar groups. For example, its engineering 
snts need more math, or its business students 
: know how tr spell. As French (1956) has 
i, the similarity score technique, when used 
unselected criterion groups, might result in 
jraging students with similar deficiencies to 
these areas. Use of regression-based success 
lates in conjunction with similarity scores 
d guard against this. In fact, Tatsuoka (1956) 
developed an index which combines success 
>imilar>ty estimates into one, overall probabili- 
he index has considerable promise for place- 
: applications. However, guidance applications 
ar limited because the joint index makes it 
issibie for a counselee to place separate values 
le success and similarity estimates. Instead, he 



is confronted with a single number somewhat 
analogous to a joint index of height and weight. 

Two additional procedures can be employed by 
the college to combat possible perpetuation of 
curricular group deficiencies through use of 
similarity scores. Both involve the strategy of 
redefining the criterion groups. The first, an 
enripirical procedure, involves the selection of 
criterion group members according to predeter- 
mined definitions of satisfactoriness. Thus, 
engineering students with poor math backgrounds 
and business students who flunk the freshman 
English course could be eliminated from the 
criterion groups. The second, ad hoc procedure for 
redefining criterion groups involves the arbitrary 
assignment of group position on the test variables 
or discriminant factors that are thought to be 
important. For example, current engineering stu- 
dents could be assigned a much higher mean on a 
math proficiency exam than they actually have. 
Or, the position of the mechanics and machine 
trade group (area B) shown in Figure 1, might be 
moved to the right on Factor 2, thus requiring 
couriselees to have a higher level of mechanical 
reasoning aptitude and scientific interest to appear 
similar. In each case, parallel adjustments could be 
made in the calculation of similarity scores. 
Further discussion of this strategy for combating 
the status quo has been provided by Rulon et al. 
(1967). 




Some Technical Considerations 



Dne obvious limitation of similarity score 
files is the difficulty in representing more than 
) tests or factor dimensions at one time, 
criminant analysis, fortunately, results in a 
uction in the number of dimensions needed to 
resent criterion group differentiation. Usually 
) dimensions are sufficient. Nevertheless, the 
ilarity score profile technique can be used with 
se factors by developing a series of profiles 
resenting group positions on the first two 
tors for successive values of the third (Prediger, 
r 0). Instances in which more than three factors 
uld be required to represent criterion group 
crimination are rare, judging from the results of 
criminant analyses reported in the literature. 

5o far, the discussion has involved only one or 
> general approaches which have been used to 
elop estimates of similarity to various groups. 
? similarity score approach illustrated above 
js* an independent estimate of a counselee's 
ilarity to each of the criterion groups under 
sideration. The second approach, which is 
3d on the maximum likelihood principle, 
vides probabilities that take into account the 
five degree of a counselee's similarity to each of 
criterion groups (Cooley & Lohnes, 1962). The 
ilting scores are given as decimal probabilities 
ch total 1.00. Thus, if five criterion groups of 
al size were involved and a counselee were 
ally similar to all five, his scores via the second 



approach would be .20 for each group. This would 
be true whether his similarity scores were all 99 or 
all 1. That is, relative degree of similarity rather 
than absolute amount is determined. 

As another example, suppose that a counselee 
had factor scores of 70 and 25 on Factors 1 and 2, 
respectively, in Figure 1. Through use of the 
maximum likelihood approach, he would receive a 
very high score for area E, possibly even higher 
than Fred Cartesian's score. This would occur 
because the counselee's relative similarity to stu- 
dents in area E is much greater than his similarity 
to students in the other areas. However, his actual 
degree of similarity is quite low, as can be seen by 
plotting his factor scores on Figure 1. Since a 
counselor (and counselee) would undoubtedly 
want to know this, the educational-vocational 
counseling applications of the maximum likelihood 
approach are limited. 

If, on the other hand, one is interested in 
allocating individuals to treatment groups so as to 
achieve some overall goal of efficiency or correct 
placement, the maximum likelihood procedure 
would be an appropriate technique to use. When 
differences in criterion group size and dispersion 
are taken into account, the predictions based on 
maximum likelihood procedures maximize the 
classification accuracy achievable by the predictor 
variables (Cooley & Lohnes, 1962). 



Recapitulation 



he following seven points have been empha- 
d in the foregoing discussion of objective 
miques for data-information conversion: 

. Information from tests, when viewed in the 
text of decision theory, can play an important 
in vocational development. 

. This role is primarily one of stimulating and 



facilitating exploratory behavior. 

3. Two data-information conversion pro- 
cedures— similarity scores and success estimates— 
are crucial to this role. 

4. On the basis of both logical and technical 
considerations, similarity scores are more appropri- 
ate than success estimates in stimulating and 
facilitating exploratory behavior. Success estimates 




represent an important consideration during the 
process of exploration. 

5. Similarity scores eliminate much of the guess- 
work inherent in test profile interpretation. 

6. Graphical procedures are available to provide 
help in understanding the reasons underlying a 
given counselee's similarity scores, thus avoiding 



the take-it-or-leave-it aspects of test interpretation 
based on similarity scores alone. 

7. These procedures can also facilitate use of test 
data to initiate changes in counselee characteristics 
and/or the characteristics of groups representing 
various choice options, rather than merely to 
represent the status quo. 



Implications 



AH of this is likely to be of little comfort to the 
conscientious counselor or personnel worker who 
has neither the time, training, nor inclination to 
become involved in data-related duties as versus 
people-related responsibilities. Few test users 
would argue about the need to strengthen the 
bridges between test scores and their meaning for 
the counselee; but how is the job to be done? 

Two major stumbling blocks to data-information 
conversion and the prerequisite validity studies are 
data collection and analysis, fields in which great 
strides have been made in the last 10 years through 
the use of computers. In addition to providing help 
with record-keeping functions, computers have 
made time-consuming and/or highly sophisticated 
data analyses economically and psychologically 
feasible. Approaches to data-information con- 
version that have been available for some time are 
now possible on a large scale. This is nowhere 
better illustrated than in the work of Project 
TALENT staff members, in particular, Cooley and 
Lohnes (1968). However, practical applications of 
Project TALENT findings are limited because of 
the multitude of measures involved and their 
unavailability to practitioners. Unless the test 



equating studies proposed by Cooley and Lohnes 
(1968) eventually are undertaken, counseling use 
of Project TALENT data will be restricted to 
special programs such as Project PLAN (Flanagan, 
1969). 

The computer-based system illustrated by the 
Cooley-Lohnes studies is generalizable to other 
settings, however. Development of such systems, 
either by educational institutions or private enter- 
prise, and provision for wide access to them are 
essential to any major improvements in test inter- 
pretation procedures. If systems for data-informa- 
tion conversion were readily available to the local 
practitioner, his only tasks would be to ask 
important questions of his data— and then help his 
counselees use the resulting information. Much of 
the work required in data preparation could be 
completed by clerical help or one of the several 
types of guidance technicians proposed by Hoyt 
(1970). 

Data-information conversion systems will never 
replace professional knowledge, judgment, and 
experience. However, they can go a long way 
toward moving test interpretation beyond the era 
of squint and tell. 






PART 2: FIELD TRIAL WITH FEEDBACK 



noted in Part 1 of this report, the vocational 
Hor who looks to empirical data for aid in 
terpretation faces several problems. One of 
ost serious is the paucity of validity data 
'ting many uses of tests in vocational 
‘ling. Equally serious, however, is the limited 
:al value of the validation studies available, 
nmonly reported, these studies provide the 
flor with no direct help in bridging the gap 
sn the test score and its meaning for the 
slee. Indeed, the counseling use of info ma- 
btained from validation analyses j s seldom 



reported in the literature. Test manuals do some- 
times present experience tables, regression equa- 
tions, or group profiles based on restricted samples. 
However, the counselor usually has only a set of 
norms, the suggestions in the test manual, and his 
own experience to guide him. Empirically based 
procedures for converting test data to readily 
usable counseling information, while available in 
theory, are largely lacking in practice. Such proce- 
dures will not remove all of the guesswork from 
interpretation. However, they can place test 
interpretation on e more objective foundation. 



Problem 



primary purpose of this study was to 
p and demonstrate a system for converting 
3ta to locally validated counseling informa- 
teactions of counselors and students to the 
this information in 12 field-test high schools 
jmmarized. In the development of the 
jter-based data-information conversion 
, heavy reliance was placed on the decision- 
id paradigm for local guidance research 
ed by Clarke, Gelatt, and Levine (1965), 
le multivariate descriptive strategies repre- 
by the work of Cooley and Lohnes (1968, 
Longitudinal validation studies involving 
tion and discriminant analyses were con- 
to provide the empirical base needed for 
formation conversion. Two conversion pro- 
s were used: experience (expectancy) tables 
nilarity (centour) scores, 
ough similarity score procedures were devel- 
10 years ago (Tiedeman, Bryan, & Rulon, 
their use in an ongoing guidance program is 
are. In Figure 2, an illustration of similarity 
is presented in conjunction with the student 
form used (with slight modifications) in 
ting potential vocational education students 
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in the field-test high schools. The similarity scores 
for Fred Cartesian, a fictitious counselee, appear on 
the "computer-printed label" to the right center of 
the form. As explained on the form, these scores 
reflect the similarity of Fred's aptitudes and 
interests to the aptitudes and interests of successful 
and satisfied students enrolled in eight vocational 
program areas. 

Previous research has shown that the relation- 
ships between antecedent and outcome variables 
required for data-information conversion are likely 
to vary from one setting to another (Bennett, 
Seashore, & Wesman, 1966; Ghiselli, 1966; Pass- 
more, 1968; Prediger, Waple, & Nusbaum, 1968). 
Hence, the nature of these relationships must be 
determined for the setting in which data- 
information conversion procedures are to be used. 
For this reason, answers to the following research 
questions were sought: 

Similarity Score 

1. Is it possible to differentiate successful and 
satisfied students enrolled in 12 vocational educa- 
tional programs through use of aptitude and 




EXPLORING PENTA-COUNTY VOCATIONAL PROGRAMS 

If you are thinking about going to Penta-County, you probably face a difficult decision— the choice of which 
vocational program you wish to enter. This report won't tell you what to do. But it will provide some information 
that can help you explore what Penta-County has to offer. The vocational programs at Penta have been grouped 
into the 12 areas listed to the left of the box below. Your counselor will give you a label that fits over the box. This 
label contains scores giving a rough estimate of your similarity to students in the different programs. These 
“similarity scores" are based on aptitude and interest tests you have taken in the last year. 



THE KEY POINT IS THIS: The higher your score for an area, the more similar you are to students in that 
vocational area. The highest score you can get is 100. The lowest score *rn. A zero score for are,. E would mean 
that your test scores do not look like the scores made by students in c a ional horticulture. It's stiff O.K. to 
consider horticulture, however. Test results, after all, don't give the whol okccuv ^ You must consider them along 
with all the other things you know about yourself and Penta-County prograrvs. 



THE BEST WAY TO USE THIS REPORT is to find the vocational programs in wrich yow score the highest. These 
are programs you might want to explore-find out more about. Perhaps you vwould net have thought of them 
otherwise. You certainly don't want to overlook a good possibility. There's to o m ~ch at stake. 



VOCATIONAL AREAS 

Mostly boys enroll 
\ A. Carpentry 

E. Auto & Ag. Mechanics, 
Machine Trades 

C. Radio & TV, Electronics 

D. Auto Body, Welding 

Both boys & girls enroll 
\ E. Vocational Horticulture 

F. Distributive Education 
V G. Commercial Art, 

Printing, Drafting 

H. Data Processing, 

Account Clerk 

Mostly girls enroll 

I. Child Care Aide or 
Ass't., Community & 
Home Service, 

Dietary Aide 

J. Cosmetology, Dental 
Assistant 

K. Co-op Office Education, 
Office Machines 

L. High Skill Steno 



SAMPLE LABEL 



032154 FRED E CARTELS! lv 10/14/69 

STUDENT SIMILARITY SCORES 'OR P-C VOC. PROGRAMS 

I J K L 



AREA= ABCDEFGH 

SC0RES= 41 14 03 26 87 25 28 01 

RANK® 13 

AREA® ABCDEFGH 



I 



K L 



PROFILE FACTOR SCORES: 56, 36 



SO HOW DO YOU USE T HE SCORES ON YOUR LABEL? 

First paste your label on the box shown above. Next, find and rank your top 3 or 4 
scores. Give the highest score a rank of 1, etc., and write the ranks on the line below 
your scores. Finally, put a check mark beside the names of the 3 or 4 areas ranking 
the highest. These are the areas that your test results suggest you might want to find 
out more about. Some students receive low scores in all of the areas. This simply 
means that the test results aren't of much help in suggesting areas to explore. 
Whether your scores are "high" or "low," your counselor can help you figure out 
why they came out the way they did. 



In order to judge how successful you might be in a program, you must also consider if you have the course work^ 
aptitudes, and personal desire .hat is needed. This report does not tell you tb at. However, with the help of Y ol ~ 
counselor and your parents, you can use it along with other information as you exolore the programs available *u 

Penta-County. 

O 

ERLC 



Fig. 2. Similarity Score Report rorm. 

1 12 13 



interest measures obtained prior to entry into 
programs? If so, 

2. Which measures are most effective and what is 
the nature of the group differentiation that is 
achieved? 

Experience Tables 

3. Within each of the vocational program areas, 
considered separately, which of the aptitude 
measures has the highest correlation with success? 



4. Is the use in vocational guidance of the best, 
two-variable combination of predictors warranted 
on the basis of the level of correlation achieved and 
the contribution made by each predictor? 

Although the answers to the previously stated 
questions are specific to the sjt. y in which the 
study was conducted, the techn ques are directly 
transferable to other settings. For example, the 
same questions could be asked f data obtained 
from students prior to entry into arious occupa- 
tional groups or college majors. 



Procedures 



A detailed description of the research design and 
statistical analyses required to answer the four 
questions on antecedent-outcome variable relation- 
ships has been presented by Prediger (1970). Only 
a summary is given here. 



Subjects and Variables 



The subjects were enrolled at the Penta-County 
Vocational High School, an area vocational school 
serving 14 feeder high schools surrounding Toledo, 
Ohio. Students entering Penta-County as juniors or 
seniors in the fall of 1966, 1967, and 1968 formed 
the sample (N = 1,584) from which various analysis 
groups were identified. Although the same schools 
were involved, the analysis sample did not overlap 
with the field-test sample. 



Scores from the following tests and inventories 
were used as antecedent variables: General 

Aptitude Test Battery, Mechanical Reasoning 
subtest of trie Differential Aptitude Tests, and the 
Kuder Preference Record— Vocational. The 20 
measures involved were generally obtained during 
the fall of the year preceding a student's entrance 
into the vocational school, although the actual time 
of testing was left to the discretion of feeder 
school counselors. Since several Kuder scales are 
subject to substantial sex differences, normalized 
standard scores based on separate percentile norms 
for males and females were used in the analyses. 

Score reports for the above measures were 
provided to feeder school counselors for use in 




helping students explore the vocational programs 
available at Penta-County. The scores were not 
used by Penta-County personnel for placement of 
students in programs. Instead, students applied for 
the programs of their choice. 

One additional antecedent variable, student GPA 
prior to entering Penta-County (PRE-GPA), was 
also available. Typically, PRE-GPA was based on 
all feeder school grades received during the fresh- 
man year and the first semester of the sophomore 
year. 

Design 

The 24 programs offered at the vocational 
school were grouped into the 12 areas listed at 
the left of Figure 2. Regression analyses required 
to answei Research Questions 3 and 4 were 
performed, separately, for each of the 12 areas. 
However, answers to Research Questions 1 and 2 
required that discriminant analyses be conducted 
across the various areas. If the analyses were 
performed on all 12 areas simultaneously, sex 
differences from area to area might obscure other 
differences among program areas. One might find 
that the interest and aptitude measures chiefly 
differentiate programs enrolling girls from those 
enrolling boys, which would be of little value 
since there are better ways to differentiate girls 
from boys. Clearly, a design was needed that 
controlled for the effect of sex differences on 
differences among programs. 
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D minimize the problem noted previously and at 
;ame tire provide for comprehensive analyses, 
12 vocational areas were organized into the 
wing groups: areas enrolling primarily males 
: 503), areas enrolling primarily females (N = 

, and areas having a substantial enrollment of 
i males and females (N = 483). As shown in 
re 2, the male, female, and mixed groups each 
jded four areas. Separate discriminant analyses 
i run on the eight male and mixeo groups 
bined (the M-MF analysis group) and the eight 
ale and mixed groups combined (the F-MF 
ysis group). Similarity scores were also based 
this grouping. Each analysis group had about 
10 members, with the mixed group appearing in 
l analysis groups. 

he vocational areas included in the dis- 



criminant analyses required by Research Questions 
1 and 2 did not include students who (a) dropped 
out of school or returned to the feeder high school, 
(b) expressed dissatisfaction with program choice, 
or (c) failed to achieve a GPA of 1 .8 (on a 4-point 
scale) in their vocational courses. In this sense, 
students in the discriminant analysis criterion 
groups can be described as "successful and 
satisfied." 

The regression analyses required by Research 
Questions 3 and 4 excluded only those students 
who left school before they had established a gvade 
record. Vocational course GPA at the time of 
graduation or dropout was used as the criterion of 
success. The median number of students in the 12 
vocational areas involved in the regression analyses 
was about 110. 



Results and Conclusions 



lultivariate analyses of variance conducted or. 
aptitude and interest measures showed that 
a types of measures, used alone cr in combina- 
i, differentiated the vocational areas far beyond 
.01 level of significance. Thus, an affirmative 
nor was obtained for Research Question 1. 
:riminant analyses were used to identify the 
ensions (factors) of test data that best differ- 
iated the areas. Since a large proportion of the 
iriminating power of the measures was concen- 
ed in the first two factors, attention is focused 
these two factors in the results following. 

"he nature of the factors that best differentiated 
vocational areas in the M-MF analysis group 
I the manner in which these areas were differ- 
iated can be seen from the "similarity score 
file" presented as Figure 1 in Part 1 of this 
ort. (A profile was also developed for the F-MF 
lysis group.) The axes of the coordinate grid, 
ich are scaled with a mean of 50 and a standard 
iation of 10 for the 8 vocational programs 
nbined, represent the two most effective dis- 
ninant factors. The position of each vocational 
a mean (centroid) on the factors has been 
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plotted as a single point, as have the factor scores 
of our fictitious counselee, Fred Cartesian. (These 
scores appear at the bottom of the label in Figure 
2.) Perspective on amount of overlap among the 
areas is provided by the ellipse enclosing the scores 
of about 50% of the members of Group E, 
vocational horticulture. 

Similarity score profiles provide a concise 
answer to Research Question 2. As noted in Part 1 
of this report, aptitude and interest measures 
having a substantial relationship with the factors 
are shown as anchors on the horizontal and vertical 
axes. In Figure 1, the masculine-mechanical areas 
(auto body, welding, carpentry, auto mechanics, 
etc.) are all relatively close together and toward the 
mechanical-outdoor interest end of Factor 1 . Voca- 
tional horticulture differs from these groups 
chiefly in its position toward the artistic interest 
end of Factor 2. The data processing and account 
clerk groups tend to be characterized by high 
clerical and computational interest, PRE-GPA, and 
clerical perception aptitude. The radio, TV, and 
electronics groups are not exceptional in these 
respects, but they do have relatively high mechan- 

15 




reasoning aptitude, scientific interests, and 
>ibly low artistic interests. The positions of 
sr areas can be analyzed 'n a similar fashion. 
Correlation analyses uung the aptitude measures 
predictors and vocational area GPA as the 
erion of success were run for each of the 12 
is. In answer to Research Question 3, the 
relations showed that PRE-GPA was, without 
eption, the best predictor across the areas. 
>ending on vocational area, a variety of other 
isures ranked second in order of effectiveness. 
i correlations obtained with PRE-GPA ranged in 
from .30 for carpentry to .70 for high skiii 
lography, with the median being about .45. 

"he most effective combination of two pre- 
:ors was determined for each vocational pro- 
n group by means of multiple correlation 
iyses. In answer to Research Question 4, practi- 
application of two-variable predictor combina- 
is was warranted in all but 2 (carpentry, and 



auto body and we(ding) of the 12 vocation areas. 
The multiple correlations ranged in size jiti .36 
for carpentry to .73 for hiigh skill stenogr y. The 
median was about .49.. There was substantial 
variation among the ^ntiitude measures umpa- 
nying PRE-GPA in the two-variable cornbir Ttions. 
For example, mechanLul reasoning was the ecomd 
variable for the auto mechanics are^, an 4 finger 
dexterity for the cosmetology area. In general, very 
little predictive ability was lost by using thee best 
combination of two predictors rather thar u. e best 
three. 

Cross-validation analyses based on fall o T 969 
entrants will be conducted when these students 
have had a chance to graduate. A follow-up study 
has been completed for 1966 and 1967 er^rrnts. 
This study, and the studies that are projected, will 
provide post-high school criteria of vocational 
development, satisfaction, and success. 



Data-lnformation Conversion 



’he results described above provide the poten- 
area vocational school student with little help 
the exploration of vocational program choice. 
:tical payoff from the statistical analyses does 
occur until the results are used to convert 
ient data into counseling information, 
n the case of success estimates, data-informa- 
i conversion was readily accomplished via 
lie-entry and double-entry experience tables, 
jle-entry experience tables showing the per- 
tage of students obtaining various vocational 
gram GPAs at various levels of PRE-GPA were 
eloped for each of the 12 vocational areas, 
jble-entry tables based on the best combination 
two predictors were developed for the 10 areas 
vhich this was warranted. Through use of these 
les, counselors and students could see the 
ure of the relationship between the most 
ictive aptitude measures and vocational program 
cess. 

►imilarity scores, the second data-informaticn 
version procedure, were based on a combina- 
l of 10 of the most effective aptitude and 



interest measures. Separate sets of similarity scores 
were developed for boys and girls. Each set 
contained scores for the eight vocational program 
areas appropriate to the student's sex. Reports 
were in the form of a computer-printed label 
attached to a preprinted interpretation sheet 
similar to the one shown in Figure 2. Emphasis in 
the use of these reports was in stimulating students 
to explore those programs for which they received 
the highest scores. Counselors also received similar- 
ity score profiles for use in representing, visually, a 
student's similarities and analyzing their basis. This 
was accomplished by plotting the factor scores 
reported for each student on the appropriate 
similarity score profile, as illustrated for Fred 
Cartesian in Figure 1. Both the amount and 
direction of difference between a student's 
position and that of the various vocational areas 
provide information relevant to vocational counsel- 
ing. A detailed description of this test interpre- 
tation procedure is provided in Part 1 of this 
report. 
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Field Tests 



Initial tie d tests of the similarity score reports 
nvolved 160 potential vocational school enrollees 
ittending four feeder high schools during the 1968- 
39 school year. Field-test counselors felt that the 
;imilarity scores were much more helpful than 
ocal norms or regular test score reports. However, 
;hey sometimes fount: that the similarity scores 
were difficult to interpret, especially when a 
jtudent's scores were all low or when a student 
questioned why his scores came out as they did. As 
a result of these reactions, similarity score profiles 
[e.g.. Figure 1) were developed to facilitate similar- 
ty score interpretation. 

Field testing during the 1969-70 school year 
involved approximately 900 potential vocational 
school enrollees attending 12 of the 14 feeder high 
schools. In addition to the commercially available 
test score reports, counselors received similarity 
score reports, a set of experience tables, a set of 
similarity score profiles, and an interpretive 
manual. Suggestions for improvement resulting 
from the second field test chiefly involved modifi- 
cations of the similarity score profiles to facilitate 
their introduction to students. The profiles initially 
provided to the counselors contained ellipses show- 
ing the distribution of students in each of the eight 
vocational areas appropriate to the student's sex. 
To reduce the visual confusion experienced by 
some students and counselors, two series (one for 
girls and one for boys) of three similarity score 
profiles were developed for field testing during the 
1970-71 school year. Figure 1 represents the 
intermediate stage in the series for boys. The first 
profile in the series contained no ellipses while the 
third profile contained ellipses for all eight areas. 
Depending on student need and understanding, any 
or all of the three profiles can be used in test 
interpretation. 

An informal survey of student reaction to 
reporting procedures was obtained after the 



1969-70 field tests. Nine of the 12 schools 
identified a "reasonably representative" sample of 
students who had received the reports. When asked 
"Did you find the Similarity Scores ... to be 
helpful as you thought about programs that you 
might enter at Penta-County?", only 13 of the 166 
students in the survey chose the response option: 
"They really weren't of any help to me." Another 
key item on the student survey was worded as 
follows: "What was the main way in which 
Similarity Scores were helpful to you? (Please 
check only one response.)" Student response per- 
centages are shown in parentheses after each of 
the following response options: 

1 . They weren't of any help. (8%) 

2. They told me which program I should enter. 

( 6 %) 

3. They suggested programs that I hadn't 
thought about before. As a result, I looked into 
some of these programs. (38%) 

4. They backed up the program choices I had 
already made. (34%) 

5. They suggested that some programs I had 
been thinking about might not be as "right" for me 
as some other programs. (13%) 

6. They told me that I shouldn't go to Penta- 
County. (1%) 

It is encouraging to note that few students 
viewed the similarity scores as telling them what to 
do and that most students saw the scores playing 
either an exploratory or a confirmatory role. 
Student comments on the reporting procedures 
were especially refreshing. Who could feel dis- 
appointed by a test interpretation that . . told a 
little more about me that I didn't quite know"! 
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Discussion 



ocational counseling has much to gain from 
availability of validated test information. The 
-information conversion procedures illustrated 
lis paper have been available for a number of 
s. Existing computer programs (Cooley & 
nes, 1962, 1971) make them feasible for any 
iseling agency or guidance department having 



access to a computer of moderate size. Regional 
services analogous to test scoring services could 
also be established. 

Local norms are now in widespread use when 
once they existed mostly as illustrations in text- 
books. Perhaps locally validated test information 
will also become commonplace. 



u ? 
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